Online Appendix for RELEAF: An Algorithm for Learning and Exploiting Relevance

This online appendix is composed of two sections. In the first section we give the proof of Theorem 5 in [1]. The second section is an extensive version of the numerical results given in Section V of [1]. I. PROOF OF THEOREM 5 A. Preliminaries Let A := |A|. We first define a sequence of events which will be used in the analysis of the regret of RELEAF. For p ∈ PR(a),t, let π(a,p) = μ(a,xR(a)(p)), where x ∗ R(a)(p) = {x ∗ i (pi)}i∈R(a) such that xi (pi) is the type i context at the geometric center of p. Let W (R(a)) be the set of Drel-tuple of types such that R(a) ⊂ w, for every w ∈W (R(a)). We have |W (R(a))| = ( D−|R(a)| 2Drel−|R(a)| ) . For a Drel-tuple of types w, let D(w, D′) be the set of D′-tuple of types whose elements are from the set D−w. For any w ∈W (R(a)) and j ∈ D(w, Drel), let INACCt(a,w, j) := { |r̄ t (pw,t,pj,t, a)− π(a,pR(a),t)| > 3 2 L √ Drel max i∈R(a) s(pR(a),t) } , be the event that the sample mean reward of action a corresponding to the 2Drel-tuple of types (w, j) is inaccurate for action a. Let